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PREFACE 

Part  of  the  Project  RAND  research  program  consists  of 
basic  studies  in  mathematics.  Including  linear  programming. 

While  the  present  Memorandum  will  be  of  particular 
interest  to  mathematical  analysts  and  programmers  concerned 
with  inventory  problems,  it  will  also  be  of  general  interest 
to  most  mathematicians  and  computer  scientists. 
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SUMMARY 

This  Memorandum  concerns  an  infinite  Markov  process  with 
a  finite  number  of  states  in  which  the  transition  probabilities 
for  each  stage  range  independently  over  sets  that  either  are 
finite  or  are  convex  polyhedra.  A  finite  computational 
procedure  is  given  for  choosing  those  transition  probabilities 
which  minimize  appropriate  functions  of  the  resulting  equilibrium 
probabilities . 
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LINEAR  PROGRAMMING  IN  A  MARKOV  CHAIN 


1 .  INTRODUCTION 

Recent  studies,  cited  below,  have  indicated  considerable 
interest  in  optimization  problems  of  a  type  formulable  as  the 
problem  of  choosing  a  set  of  distributions,  constituting  the 
transition  probabilities  of  a  finite  Markov  process,  in  such 
a  way  as  to  minimize  certain  "costs"  associated  with  the 
process . 

The  following  Inventory  problem  is  a  typical  example  of 
this  class:  Let  the  n  attainable  levels  of  the  inventory 
of  an  item  constitute  the  n  states  of  a  Markov  process. 
Transition  from  one  state  to  another  will  occur  at  the  end  of 
each  of  an  infinite  sequence  of  time  periods.  Owing  to  the 
uncertain  nature  of  supply  and  demand  for  the  item  (only  its 
distributions  are  assumed  known),  the  effect  of  a  given 
Inventory  policy  must  be  described  as  a  distribution.  For  any 
inventory  policy  the  probability  p^  of  transition  from 
state  i  to  state  J  in  one  time  period  is  known,  as  well 
as  the  cost  c.  .,  dependent  on  the  policy,  which  will  be 
incurred  if  that  transition  is  made.  Under  any  policy  the 
time  series  of  inventory  levels  constitutes  a  Markov  process 
described  by  the  given  probabilities.  When  an  initial  state 
for  the  first  period  has  been  given,  the  long-run  probabili¬ 
ties  p  are  then  determined.  Intuitively,  Pj.  is  the 
probability  that,  at  a  typical  time  period  in  the  indefinite 
future,  the  transition  from  state  i  to  state  j  will  take 
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place.  The  "long-range  expected  cost"  of  using  the  particular 
policy  is  then  defined  as  2^  j  c^j  Pij*  Then  the  computa¬ 
tional  problem  is  that  of  minimizing  this  expected  cost  over 
all  available  inventory  policies. 

The  formulation  of  such  a  problem  as  one  of  linear 
programming  has  been  done  by  Marine  [5]>  d'Epenoux  [3],  and 
Oliver  [6]  for  problems  in  which  the  transition  probabilities 
p^j  can  be  chosen,  for  each  i  separately,  from  a  given 
finite  set  of  distributions.  The  same  assumption  concerning 
the  available  distributions  is  made  by  Howard  [4]  in  his 
"dynamic  programming"  treatment  of  this  class  of  problems. 

When  the  problem  is  formulated  as  a  linear  program,  however, 
it  can  be  efficiently  attacked  by  means  of  a  specialization 
of  the  decomposition  algorithm  for  linear  programming  [2]. 

This  makes  it  possible  to  broaden  considerably  the  class  of 
problems  that  can  be  handled,  by  permitting  other 
descriptions  of  the  sets  of  available  alternatives.  In  the 
sequel  two  extreme  cases  are  considered:  on  the  one  hand, 
the  case  described  above;  on  the  other,  the  case  in  which  the 
distributions  that  may  be  used  are  restricted  only  in  that 
they  must  satisfy  certain  linear  inequalities.  Since  these 
two  extreme  cases  are  handled  by  essentially  the  same  method, 
any  intermediate  cases  of  practical  interest  can  readily  be 
treated  by  the  same  technique. 
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2.  THE  PROBLEM 

Throughout  this  paper  n  is  a  fixed  integer.  By 
distribution  we  shall  mean  an  n— vector  x  *  (x^,...,x  ) 
such  that  >  0  (all  1)  and  Z  »  1.  A  Markov  process 
is  defined  by  n  distributions  P^  -  (p^,  . . .  ,pln) ,  for 
i  *  l,...,n,  where  p^  is  the  probability  of  transition 
from  state  i  of  the  process  to  state  j. 

In  the  problem  studied  here,  a  particular  Markov  process 
is  defined  by  a  choice  of  distributions  from  certain  sets 
(which  will  be  assumed  finite  in  this  section  and  the  next): 

For  each  i  «  l,...,n,  let  be  a  finite 

set  of  distributions. 

In  addition,  a  "cost"  c1(P)  is  associated  with  each  distri¬ 
bution  P  in  S^: 

For  each  1  -  l,...,n,  let  c^  be  a  real- 
valued  function  on  S^. 

For  P  in  c1(P)  is  thought  of  as  a  fee  to  be  paid  for 

the  use  of  the  distribution  P  when  passing  through  state  i. 

The  particular  manner  in  which  and  c^  are 

described  is  not  of  great  importance  in  the  discussion  which 
follows,  but  it  does  play  an  important  role  in  the  computational 
algorithm  of  Secs.  4  and  5.  The  more  extensive  discussion  of 

*The  symbol  "Z"  is  used  throughout  as  an  abbreviation 
n 

of  "  Z  ". 

1-1 
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Sec.  5  can  be  anticipated  by  the  observation  that  the 
algorithm  is  aimed  at  handling  either  of  the  following  two 
extremes:  (a)  is  given  as  an  arbitrary  finite  set,  and 

c1  as  an  arbitrary  function  on  (b)  a  finite  set  of 

linear  inequalities  in  n  +  1  variables  is  given,  defining 
an  (n  +  l) -dimensional  polyhedron  in  such  a  way  that  the  first 
n  coordinates  of  any  point  of  this  polyhedron  form  a 
distribution.  The  first  n  coordinates  of  any  extreme  point 
of  this  polyhedron  constitute  a  member  P  of  with 

c±(P)  defined  as  the  minimal  (n  +  l)— st  coordinate  of  all 
extreme  points  whose  first  n  coordinates  constitute  P.* 

If  now  particular  P^  in  are  chosen  for  each  i, 

then  a  Markov  process  is  defined.  Let  x  be  an  equilibrium 
distribution  for  this  process — that  is,  a  distribution  satis¬ 
fying  relation  (2.2)  below.  The  "expected  cost"  of  the 
process  per  stage,  when  the  equilibrium  x  obtains,  is  then 

(2.1)  Z  c1(P1)  xr 

The  Markov  programming  problem  is  that  of  choosing  the  p±  in 
such  a  way  that  this  expected  cost  is  minimized. 

'  5 

It  will  be  seen  from  the  discussion  of  case  (b)  in  Sec.  5 
that  the  restriction  of  to  extreme  points  of  the  poly¬ 

hedron  is  unnecessary,  since  even  if  all  points  were  admitted, 
only  extreme  points  would  appear  in  the  solution  of  the 
problem.  This  restriction  is  made  because  of  the  convenience 
of  assuming  SA  to  be  finite. 
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Formally  stated,  the  problem  is  as  follows: 

Determine  p±  in  S±  (l-l,...,n)  such 
that  (2.1)  Is  minimized  for  all  x  for  which 

(2.2)  2  0,  Z  xi  -  1,  and  Z  x±  PA  -  x. 

It  will  be  convenient  for  the  sequel  to  restate  this 
problem  in  such  a  way  that  the  equations  (2.2)  have  constant 
right-hand  3ides. 

For  each  i,  let  T1  be  the  set  of  all  n-vectors 
(2-3)  -  (Pn»...,p11  -  1^-*-»Pln) 

for  which  (pn,  .  .  -»Pln)  -  P1  is  in  S1#  and  define  c±  on 

^i  ^  ci^i^  "  ci^l^’  using  the  correspondence  given.  The 
problem  may  then  be  stated: 

Determine  Ql  in  ( 1  -  1 . n)  such  that 

(2.4)  Z  c1(Q1)  Xl 

is  minimized  for  all  x  for  which 

(2.5)  2  O'  £  ■  1,  and  Z  x^  ■  0. 

It  is  clear  that  any  solution  x;  P,,...,Pn  of  the  problem 
stated  by  (2.1)  and  (2.2)  gives  a  solution  x;  Q1,...,Qri  of 
the  problem  (2.4,  2.5),  and  vice  versa. 


3.  FORMULATION  AS  A  LINEAR  PROGRAMMING  PROBLEM 

The  problem  (2.4,  2.3)  will  be  solved  with  the  devices 
developed  for  the  "decomposition"  of  linear  programming 
problems  of  special  structure  [2],  specialized  to  the  case 
at  hand.  The  central  idea  of  this  approach  is  the  formulation 
of  the  problem  as  one  of  linear  programming  in  which  the  data 
consist  primarily  of  the  coordinates  of  points  of  the  set  T^. 
This  will  be  done  in  this  section.  For  each  1,  let  the 

Jr 

points  of  T^  be  Q1  for  k  =■  l,...,Ki.  As  an  abbreviation, 
let  cik  -  c^(Q^)  for  all  l,k.  Consider  the  linear  program¬ 
ming  problem: 

Minimize 

(3-1)  z  clk  ylk 


urc.er  the  constraints 


(5-2) 


lk 


2  0,  Z  Zk^  y^k 


K, 


1,  Z  Z, 


:i  ^ik  *1 


Theorem  1  below  will  show  that  this  problem  is  equivalent 
to  the  problem  of  the  previous  section.  In  general,  replacing 
a  discrete  problem  by  a  continuous  one  in  this  manner  can 
lead  to  a  solution  that  is  not  discrete.  The  Lemma  below 
shows,  however,  that  for  the  problem  studied  here  the  solution 
of  the  continuous  problem  is  Itself  sufficiently  "discrete"  to 
ensure  equivalence:  For  each  1,  only  a  single  Qi  is 
actually  involved  in  the  solution  of  the  problem  (3.1,  3-2). 
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Lemma .  There  Is  a  solution  of  the  problem  (3.1,  3.2) 
with  the  property  that  for  each  1  there  Is  at  most  one  k 
for  which  yiJc  >  0. 


Proof.  The  coefficients  and  right-hand  side  of  the 
linear  programming  problem  (3-1,  3-2)  are  displayed  in  the 
table  below,  headed  by  their  variables  y^,  where  p 
denotes  the  appropriate  component  of  the  distribution  P 
corresponding  to  Q^. 


ij 


Table  1 

COEFFICIENTS  OF  THE  LINEAR  PROGRAMMING  PROBLEM 


1 

0 

0 

0 


It  is  a  basic  property  of  linear  programming  problems 
[l]  that,  when  a  solution  exists,  there  is  a  solution  having 
exactly  r,  say,  positive  components  for  which  the  submatrix 
consisting  of  those  columns  of  the  coefficient  matrix 
associated  with  the  positive  components  has  rank  r.  For 
this  problem,  denote  by  B  the  (n  +  1)  x  r  submatrix  of 


(^•3)  given  by  that  property;  the  associated  solution  will  be 
the  one  whose  existence  the  lemma  asserts.  (As  a  matter  of 
fact,  the  simplex  method  solution  of  this  linear  programming 
problem  will  yield  a  solution  of  Just  this  type.) 

Let  s  be  the  number  of  rows  of  B  in  which  may  be 
found  an  entry  of  the  form  p*  1  -  1.  Excluding  the  first 
row,  the  other  n  —  s  rows  have  only  nonnegative  entries; 
since  their  right-hand  sides  are  zero  and  their  variables 
yik  Posltive,  these  rows  must  in  fact  vanish.  Hence  B  has 
Just  s  +  1  nonvanishing  rows.  The  nonvanishing  rows  are, 
however,  linearly  dependent  (the  sum  of  all  rows  but  the 
first  is  zero),  whence  the  rank  of  B  is  at  most  s,  that 
is,  s  >  r.  Since  B  has  Just  r  columns,  it  follows  that 
at  most  one  entry  of  the  form  p^  -  1  can  be  found  in  any 
row  of  B,  so  that  at  most  one  column  of  (3.3)  can  be  found 
in  B  for  J  given,  which  proves  the  lemma. 


Theorem.  The  programming  problems  (2.4,  2.3)  and  (3-1, 

J>-2)  are  equivalent,  with  solutions  related  in  the  following 
w££: 

—  v-en  y ik  gpivlng  (5*1,  3.2)  and  satisfying  the 
conclusion  of  the  lemma,  for  each  1  let 


*i 

xi 

*i 


vlk  I 

.  <jk  ?  ylk  >  0 

•  0 

arbitrary  in 


for  some  k, 


yik  ”  0  Lor  all  k. 
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On  the  other  hand,  given  x±,  Q  solving  (2.4,  2.5), 

let 


ylk 


for  k  such  that  =*  q 
0  otherwise . 


The  proof  is  obvious. 


4.  COMPUTATIONAL  ALGORITHM — THE  MASTER  PROBLEM 

The  linear  programming  problem  formulated  in  the  last 
section  has  only  n  +  1  equations,  but  it  has  Z 
variables,  a  number  which  may  be  very  large,  and  in  fact  not 
even  known  for  problems  whose  data  are  given  implicitly.  The 
revised  simplex  method  [l]  is  particularly  advantageous  for 
problems  having  many  more  variables  than  constraints.  The 
decomposition  algorithm  uses  this  efficiency  of  the  revised 
simplex  method  by  clearly  separating  the  considerations 
involving  the  constraints  alone  from  those  connected  with  the 
handling  of  the  variables.  That  part  of  the  problem  involving 
the  constraints  is  called  the  "master  problem,"  and  its 
handling  is  set  forth  in  this  section.  That  part  of  the 
problem  involving  the  variables,  called  the  "subproblem,"  is 
dealt  with  in  the  next  section.  It  will  be  seen  that  the  work 
of  treating  the  master  problem  consists  of  little  more  than 
the  application  of  the  revised  simplex  method  to  the  Markov 
programming  problem  as  formulated  in  Sec.  5.  The  general 
iterative  step  is  given  below,  followed  by  the  procedures  for 
initiating  the  Iterative  process  and  for  passing  from  the 
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determination  of  an  initial  feasible  point  (Phase  One)  to  tne 
determination  of  the  solution  of  the  problem  (phase  Two). 

(The  phenomenon  of  degeneracy  plays  the  same  role  in  this 
algorithm  as  in  any  linear  programming  problem,  and  it  will 
be  supposed  that  standard  methods  [l]  may  be  relied  upon  when 
necessary . ) 

The  Iterative  Step 

At  any  step  in  the  course  of  the  solution  of  the  problem 
(3.1,  3.2)  by  the  revised  simplex  method,  there  will  be  at 
hand  some  n  +  1  column  vectors  Q  ,...,Q  (of  length 
n  +  l)  constituting  a  "feasible  basis";  that  is,  they  are 
linearly  independent,  and  the  right-hand  side  of  the  equations 
(30)  may  be  expressed  as  a  nonnegative  linear  combination  of 
them.  (The  weights  in  this  linear  combination,  which  of 
course  constitute  a  solution  of  equations  similar  to  (3. 3) 
deriving  their  coefficients  from  the  Q1,  are  called 
collectively  a  "basic  feasible  point.") 

Let  the  "cost"  c1  be  associated  with  the  column 
for  1  -  l,...,n  +  1.  The  "prices,"  assumed  known, 
associated  with  this  basis  are  defined  to  be  the  components  of 
the  (n  +•  l)— vector  v  =■  (u^,  . .  .  >*Ml)  satisfying  the  relation¬ 
ships  irQ*  -c*  (i«l,...,n  +  l). 

One  Iteration  of  the  simplex  method  consists  of  the 
following  steps: 

(1)  Find  a  column  Q  of  the  matrix  (3.3)  which,  with 
its  associated  cost  c,  satisfies  the  relation 
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(4.1)  c  —  ttQ  <  0. 

Ccnr..only,  the  column  chosen  is  that  for  which  c  -  7rQ  is 
minimal.  (This  is  the  only  point  in  the  revised  simplex 
method  at  which  all  the  columns — i.e.,  all  the  variables — in 
the  problem  come  into  play.  This  step  forms  the  "subproblem," 
which  will  be  discussed  in  Sec.  5.) 

(2)  If  no  column  satisfying  (4.1)  can  be  found,  then  the 
current  bu3ls  is  optimal,"  and  the  solutions  of  the  equations 
(3-3)  solve  the  linear  programming  problem. 

(3)  Otherwise,  add  the  column  found  in  Step  1  to  the 
current  basi3,  and  remove  one  column  in  such  a  way  (given  by 
the  rules  of  the  simplex  method)  that  the  remainder  still 
forms  a  feasible  basis.  Calculate  the  new  prices,  and  begin 
again. 

Phase  One 

The  algorithm  can  be  started  with  precisely  the  same 
device,  called  Phase  One,  used  for  the  general  linear  program¬ 
ming  problem.  This  device  consists  in  augmenting  the  problem 
with  m  +  1  "artificial"  variables  in  terms  of  which  an 
initial  feasible  basis  and  the  prices  associated  with  the 
corresponding  initial  feasible  basis  are  readily  given.  The 
algorithm  can  then  be  applied  to  the  problem  of  removing  the 
artificial  variables.  After  this  has  been  done,  the 
required  starting  conditions  for  the  ordinary  application  of 
the  algorithm  are  automatically  met. 
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°r  =  *  •••»n  +  1,  let  yA  be  a  nonnegative  variable 

let  I±  be  the  i-th  column  of  the  (n  +  l)-^rder  identity 

matrix;  and  let  c±  -  1  be  the  cost  associated  with  the 
variable  y±.  For  this  phase,  replace  all  the  costs  c 

ik 

of  the  original  problem  with  zeroes  . 

Designating  lr-..,In+1  as  the  initial  feasible  basis, 
employ^ the  iterative  step  outlined  above  until  the  linear 
form  ^  y±  has  been  minimized.  (Note  that  the  initial 
feasible  point  is  (y,, . . . ,yn+1)  =  (1,0,. ..,0)  and  that  the 
initial  prices  are  v  =  ( 1, 1 , . . . , 1)  .  ) 

The  above  process  will  reduce  the  form  Z  y±,  and 
hence  each  y±  separately,  to  zero.  (if  lt  did  not,  then 
the  equations  (3.2)  would  have  no  solution,  which  is 
impossible.)  owing  to  the  linear  dependence  of  the  equations 
(3-2),  some  of  the  starting  columns  will  remain  in  the 

feasible  basis  at  the  end  of  Phase  One;  this  can  be  shown  to 
cause  no  difficulty  in  the  ensuing  process  [1). 

Phase  Two 

When  Phase  One  is  finished,  restore  the  deleted  costs 
clk  to  the  col^nns  qJ,  using  these  costs  from  now  on  in 
the  determination  of  the  prices  tt.  Repeat  the  iterative 
step  until  it  terminates  in  its  part  (2). 

At  termination,  associated  with  each  Q*  m  the  final 
feasible  basis  is  a  component  of  the  "feasible  point,"  the 
weight  given  Q*  in  expressing  the  right-hand  side  of  the 
equations  as  a  linear  combination  of  the  columns  of  the  basis. 
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Por  i  *  according  to  the  Theorem  of  Sec.  3,  there 

can  be  no  more  than  one  In  Ti  in  the  basis  having 

positive  weight;  thus  let 

Jweight  for  Q^,  if  positive, 
xi  I 

10,  otherwise. 

The  resulting  (x1,...,xn)  is  the  solution  of  the  problem 

(2.2). 

5-  THE  SUBPROBLEM  AND  PROOF  OF  TERMINATION 

As  mentioned  previously,  a  detailed  discussion  of  part 
(1)  of  the  iterative  step  of  Sec.  4,  the  "pricing  out" 
operation  in  the  ordinary  revised  simplex  method,  will  be 
given  in  this  section.  Given  the  quantities  tt,  the  problem 
is  to  determine  some  column  Q  and  its  associated  cost  c 
for  which 

(5.1)  c  -  ttQ  <  0, 

where  Q  may  come  from  any  of  the  sets  T^ .  How  this  is 
done  depends  on  the  nature  of  the  description  of  the  original 
sets  Sj^  from  which  the  ^  were  obtained.  It  is  evident 
that  the  problem  of  satisfying  (5-1)  from  among  the  union  of 
all  the  Ta  may  be  "decomposed"  into  n  problems,  the  i-th 
one  of  which  (i  «  l,...,n)  is  that  of  satisfying  (5.1)  for 
Q  in  Ti*  If  all  of  these  "subproblems"  can  be  solved,  then 
the  stated  problem  has  been  solved. 
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For  each  1  ■  l,...,n,  one  of  the  two  "extreme"  cases 
(a)  or  (b)  mentioned  in  Sec.  2  may  obtain.  (Some  "intermediate" 
case  might  also  be  considered,  but  this  will  not  be  done  here.) 

(a)  Sj^  is  given  directly  as  a  finite  set  of  distributions, 
a  cost  c1(P)  being  associated  with  each  member  P  of  Si . 

(b)  There  is  given  a  finite  set  of  linear  relations 

(5*2)  Sj(z)  £  0,  J  =  1, . . . ,m, 

in  the  n  +  1  variables  ( zi> • • • » zn+i)  a  z>  such  that  if  z 
satisfies  (5.2),  then  (z^,...,zn)  is  a  distribution;  S.^  is 
defined  to  be  the  set  of  all  P  -  (zlf...,z  )  such  that  for 
3ome  zn+1J  z  =  (P»zn+i)  is  an  extreme  point  of  the  set  of 
all  z  satisfying  (5*2);  and  for  p  in  ci(P)  is 

defined  to  be  the  smallest  value  of  z  for  which  (P;z  ) 

i  n+jL 

is  such  an  extreme  point.  (The  index  1  has  been  omitted 
above;  of  course,  the  relations  (5-2)  may  be  different  for 
each  i,  or  even  absent.) 

For  the  case  (a),  there  is  not  much  to  be  said.  Phrased 
via  the  definition  (2.5)  in  terms  of  relation  (5.1) 

urges  the  selection  of  P  in  S±  for  which 

(50)  ci(P)  -  ttP  +  Tr±  <  0. 

Such  a  P  will  yield  through  (2  0)  a  column  Q  satisfying 
(5.1). 

Case  (b)  is  more  Interesting,  in  view  of  the  fact  that 
the  extreme  points  of  the  polyhedron  defined  by  (5.2)  have  not 
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been  assumed  to  be  available  in  advance.  Replacing  p  and 
c1(P)  in  (5*5)  by  their  definitions  in  this  case,  it  is 
desired  to  choose  z  =  ( z^ , . . . , zn+1 )  under  the  constraints 
(5*2)  in  such  a  way  that 

n 


(5-4) 


Jn+1 


Z  7T  .  Z  ,  +  7T,  <  0. 
J=1  J  J  1 


This  is  nearly  a  linear  programming  problem.  If  the 
customary  procedure  for  the  simplex  method — that  of  making 
the  left-hand  side  of  (5-5)  as  small  as  possible — is 
followed,  then  the  task  is  precisely  a  linear  programming 
problem:  Under  the  constraints  (5-2),  minimize  the  left-hand 
side  of  (5*4).  Once  performed,  if  the  result  is  not  negative, 
this  minimization  is  of  no  interest;  but  if  it  is  negative, 
then  the  co lumn  Q  *  (z^,...,z^  —  1 ,  . . . , z^ )  and  its  cost 
c  =  zn+1  constructed  from  the  solution  of  the  problem 
satisfy  equation  (5.1).  Furthermore,  Q  will  be  an  extreme 
point  of  the  polyhedron. 

The  complete  solution  of  the  subproblem  then  goes  as 
follows:  For  each  i  =  l,...,n,  attempt  to  satisfy  (5*1) 
from  T^  —or  equivalently,  attempt  to  satisfy  (5*5),  or 
(5*4),  from  S^.  If  this  can  be  done  for  any  i,  part  (1) 
of  the  iterative  step  of  Sec.  4  can  be  accomplished.  (It  Is 
Indifferent  to  the  fact  of  the  convergence  of  the  procedure, 
although  probably  not  to  its  rate,  whether  or  not  the  I  for 
which  (5*1)  is  minimized  is  chosen.)  If,  on  the  other  hand, 
(5*1)  cannot  be  accomplished  for  any  i,  then  part  (2)  of  the 
iterative  step  obtains,  and  the  procedure  has  terminated. 
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It  remains  only  to  show  that  the  algorithm  is  finite. 
This  follows  immediately,  however,  from  the  finiteness  of  the 
simplex  algorithm  for  linear  programming  [l],  for  as 
described  in  Sec.  4,  this  algorithm  is  precisely  the  simplex 
method  applied  to  the  linear  programming  problem  (3.1,  3.2). 
Whether  the  sets  of  columns  are  described  in  the  manner 

(a)  or  (b)  above,  they  are  finite  in  number,  and  the  proof 
is  complete. 
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